Towards more effective testing of communications-critical large scale systems

....................................................................................................................................................... 2 ACKNOWLEDGEMENTS .................................................................................................................................... 6 GLOSSARY ........................................................................................................................................................ 7 1. CHAPTER 1: INTRODUCTION AND OVERVIEW ......................................................................................... 9 1.1 CURRENT TEST THEORY AND PRACTICE ............................................................................................................ 11 1.2 DIRECTION AND CONTRIBUTION OF THIS RESEARCH ............................................................................................ 16 2. CHAPTER 2: LITERATURE REVIEW ......................................................................................................... 18 2.1 LITERATURE REVIEW INTRODUCTION ............................................................................................................... 18 2.2 WHAT IS SOFTWARE TESTING ......................................................................................................................... 18 2.3 RELATED TERMINOLOGY ................................................................................................................................ 19 2.4 GOALS OF SOFTWARE TESTING ....................................................................................................................... 21 2.5 HOW IS SOFTWARE TESTING DONE? ................................................................................................................ 22 2.6 THE STRUCTURE AND SETUP OF A TESTING ORGANISATION ................................................................................... 23 2.7 TESTING STANDARDS AND METHODS ............................................................................................................... 24 2.8 DIFFERENT TYPES AND LEVELS OF TESTING ........................................................................................................ 31 2.9 TESTING DOMAINS POTENTIALLY RELEVANT TO THIS THESIS .................................................................................. 32 2.10 DISCUSSION ABOUT TESTING DOMAINS RELEVANT TO THIS THESIS ......................................................................... 34 2.11 SOFTWARE TESTING ROADMAPS ..................................................................................................................... 36 2.12 LARGE SCALE SYSTEMS RESEARCH IN THE UK ................................................................................................... 39 2.13 THE ZACHMAN FRAMEWORK ......................................................................................................................... 41 2.14 AN EXAMPLE OF TESTING PRACTICE IN INDUSTRY ................................................................................................ 43 2.15 SIGNIFICANT AREAS OF TESTING OUTSIDE THE DIRECT SCOPE OF THIS THESIS ............................................................ 44 2.16 CONCLUSION .............................................................................................................................................. 49 3. CHAPTER 3: RESEARCH METHODOLOGY ............................................................................................... 51 3.1 OVERVIEW ................................................................................................................................................. 51 3.2 INTRODUCTION ........................................................................................................................................... 53 3.3 THE RESEARCH OBJECTIVE ............................................................................................................................. 55 3.4 THE CASE STUDY QUESTIONS .......................................................................................................................... 55 3.5 PURPOSE OF THE CASE STUDY......................................................................................................................... 56 3.6 CASE STUDY TYPE ........................................................................................................................................ 56 M. Nabulsi Thesis 2014 BURA Open Access Version 2 Page 4 of 260 3.7 THE FIRECONTROL CASE STUDY ...................................................................................................................... 58 3.8 THE CASE STUDY DESIGN ............................................................................................................................... 60 3.9 VALIDITY .................................................................................................................................................... 71 3.10 FURTHER DISCUSSIONS REGARDING RESEARCH METHODOLOGY ............................................................................. 73 3.11 SUMMARY DIAGRAM ................................................................................................................................... 74 4. CHAPTER 4: THE PROPOSED NEW TEST FRAMEWORK .......................................................................... 75 4.